Image projection system and control method for image projection system

ABSTRACT

An image projection system includes an image projecting section configured to project an image onto a projection surface, a control section configured to cause the image projecting section to project a pattern image, an imaging section configured to capture the pattern image projected on the projection surface, a detecting section configured to detect a plurality of reference points on the basis of the pattern image captured by the imaging section, and an image-information correcting section configured to correct, on the basis of positions of the reference points detected by the detecting section, the image projected by the projecting section. The pattern image includes a plurality of unit patterns for specifying the reference points. The plurality of unit patterns include unit patterns of seven colors.

BACKGROUND 1. Technical Field

The present invention relates to an image projection system thatprojects an image onto a projection surface and a control method for theimage projection system.

2. Related Art

There has been proposed a technique for, in a system that projects animage onto a projection surface, which is not a simple plane, that is, aprojection surface having a three-dimensional shape, projecting apredetermined pattern image onto the projection surface from a projectorand capturing the predetermined pattern image with a camera to therebycorrect distortion of an image due to the three-dimensional shape of theprojection surface (see, for example, JP-A-2016-178448 (PatentLiterature 1)). In the projection system described in Patent Literature1, a lattice-like pattern image on which white rectangular patterns andblack rectangular patterns are regularly arrayed is used.

However, because a parallax is present between the projector and thecamera, when relatively large unevenness is present on the projectionsurface, a part of the pattern image cannot be captured by the cameraand correct correction cannot be performed. In particular, in a form inwhich correction is performed on the basis of the positions of aplurality of reference points (e.g., intersections of a lattice), if apart of the reference points cannot be detected, a correspondencerelation between detected reference points and original positions(positions on the pattern image) of the detected reference points isunclear. Therefore, correct correction cannot be performed in the entireimage. Even if the projection surface is a plane, when a part of thereference points cannot be detected because of a pattern or a stain onthe projection surface, an obstacle, or the like, the same problem couldoccur.

SUMMARY

An advantage of some aspects of the invention is to solve at least apart of the problems described above, and the invention can beimplemented as the following forms or application examples.

Application Example 1

An image projection system according to this application exampleincludes: a projecting section configured to project an image onto aprojection surface; a control section configured to cause the projectingsection to project a pattern image; an imaging section configured tocapture the pattern image projected on the projection surface; adetecting section configured to detect reference points on the basis ofthe pattern image captured by the imaging section; and a correctingsection configured to correct, on the basis of positions of thereference points detected by the detecting section, the image projectedby the projecting section. The pattern image includes a plurality ofunit patterns for specifying the reference points. The plurality of unitpatterns include at least three types unit patterns which at least oneof colors and patterns are different.

With the image projection system, because the pattern image includesunit patterns of at least the three types, at least one of the colorsand the patterns of which are different, it is possible to increase aninterval of arrangement of unit patterns of the same type compared withwhen unit patterns of two types like a white rectangular pattern and ablack rectangular pattern are used. Therefore, even when a part of thereference points are not detected by the detecting section, it ispossible to identify the detected reference points on the basis of thecolors and the patterns of the unit patterns. As a result, acorrespondence relation between the positions of the detected referencepoints and original positions (positions on the pattern image) of thedetected reference points is clarified. Therefore, it is possible toappropriately correct the image.

Application Example 2

In the image projection system according to the application example, theat least three types unit patterns may have colors different from oneanother.

With the image projection system, even when a part of the referencepoints are not detected by the detecting section, it is possible toidentify the detected reference points on the basis of the colors of theunit patterns.

Application Example 3

In the image projection system according to the application example, theat least three types unit patterns may have patterns different from oneanother.

With the image projection system, even when a part of the referencepoints are not detected by the detecting section, it is possible toidentify the detected reference points on the basis of the patterns ofthe unit patterns.

Application Example 4

In the image projection system according to the application example, theunit patterns are patterns of a shape having vertexes, and the detectingsection may detect the vertexes of the unit patterns as the referencepoints.

With the image projection system, because the detecting section detectsthe vertexes of the unit patterns as the reference points, it ispossible to easily detect the reference points.

Application Example 5

In the image projection system according to the application example, theunit patterns may be patterns having luminance distributions, and thedetecting section may detect the reference points on the basis ofluminance of the unit patterns.

With the image projection system, because the reference points aredetected on the basis of the luminance of the unit patterns, it ispossible to accurately detect the reference points.

Application Example 6

In the image projection system according to the application example, thepattern image may include a plurality of basic pattern groups in whichthe at least three types unit patterns are arranged in a predeterminedarray.

With the image projection system, because the pattern image isconfigured by the plurality of basic pattern groups, it is possible toreduce necessary types (colors and patterns) of the unit patterns. It iseasy to identify the unit patterns.

Application Example 7

In the image projection system according to the application example, inthe pattern image, the unit patterns maybe arranged such that thereference points are located along a first epipolar line determined onthe basis of a disposition relation between the projecting section andthe imaging section, and the detecting section may detect, from theimage captured by the imaging section, the reference points along asecond epipolar line corresponding to the first epipolar line.

With the image projection system, because the detecting section iscapable of detecting the reference points along the second epipolarline, it is easy to detect the reference points.

Application Example 8

In the image projection system according to the application example, theunit patterns may be belt-like patterns extending in a directioncrossing a first epipolar line determined on the basis of a dispositionrelation between the projecting section and the imaging section, and thedetecting section may detect, from the image captured by the imagingsection, the reference points along a second epipolar line correspondingto the first epipolar line.

With the image projection system, because the detecting section iscapable of detecting the reference points along the second epipolarline, it is easy to detect the reference points.

Application Example 9

In the image projection system according to the application example, thedetecting section may detect end portions of the unit patterns as thereference points.

With the image projection system, because the detecting section detectsthe end portions of the unit patterns as reference points, it ispossible to easily detect the reference points.

Application Example 10

In the image projection system according to the application example, theunit patterns may be patterns having luminance distributions, and thedetecting section may detect the reference points on the basis ofluminance of the unit patterns.

With the image projection system, because the reference points aredetected on the basis of the luminance of the unit patterns, it ispossible to accurately detect the reference points.

Application Example 11

In the image projection system according to the application example, theprojecting section and the imaging section may be integrally configured.

With the image projection system, because the projecting section and theimaging section are integrally configured, it is unnecessary to adjustdisposition of the projecting section and the imaging section everytime.

Application Example 12

A control method for an image projection system according to thisapplication example includes: projecting a pattern image from aprojecting section configured to project an image onto a projectionsurface; capturing the pattern image projected on the projectionsurface; detecting reference points on the basis of the captured patternimage; and correcting, on the basis of positions of the detectedreference points, the image projected by the projecting section. Thepattern image includes a plurality of unit patterns for specifying thereference points. The plurality of unit patterns include at least threetypes unit patterns which at least one of colors and patterns aredifferent.

With the control method for the image projection system, because thepattern image includes unit patterns of at least the three types, atleast one of the colors and the patterns of which are different, it ispossible to increase an interval of arrangement of unit patterns of thesame type compared with when unit patterns of two types such as a whiterectangular pattern and a black rectangular pattern are used. Therefore,even when a part of the reference points are not detected by thedetecting section, it is possible to identify the detected referencepoints on the basis of the colors and the patterns of the unit patterns.As a result, a correspondence relation between the positions of thedetected reference points and original positions (positions on thepattern image) of the detected reference points is clarified. Therefore,it is possible to appropriately correct the image.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described with reference to the accompanyingdrawings, wherein like numbers reference like elements.

FIG. 1 is a block diagram showing a schematic configuration of aprojector.

FIG. 2 is a diagram showing a pattern image for correcting distortion ofan image.

FIG. 3 is a flowchart for explaining the operation of the projector whendistortion correction is performed.

FIG. 4A is a plan view of a vertically disposed projection surface fromabove.

FIG. 4B is a diagram showing a captured image obtained by imaging, withan imaging section, a projection surface on which the pattern image isprojected.

FIG. 4C is a diagram showing a correction region formed in an imageforming region.

FIG. 4D is a diagram showing a state in which a display image isprojected onto the projection surface in a distortion corrected state.

FIG. 5A is a plan view of the vertically disposed projection surfaceviewed from above.

FIG. 5B is a diagram showing a captured image obtained by imaging, withthe imaging section, the projection surface on which the pattern imageis projected.

FIG. 5C is a diagram showing a correction region formed in an imageforming region.

FIG. 5D is a diagram for explaining correction processing by animage-information correcting section.

FIG. 5E is a diagram showing a state in which a display image isprojected onto the projection surface in a distortion corrected state.

FIG. 6 is a diagram for explaining the correction processing by theimage-information correcting section.

FIG. 7 is a diagram showing a pattern image in a second embodiment.

FIG. 8 is a diagram showing a pattern image in a third embodiment.

FIG. 9 is a diagram showing a pattern image in a fourth embodiment.

FIG. 10 is an explanatory diagram for explaining an epipolar geometryconcerning an image projecting section and an imaging section.

FIG. 11 is a diagram showing a pattern image in a fifth embodiment.

FIG. 12 is a diagram showing a pattern image in a sixth embodiment.

DESCRIPTION OF EXEMPLARY EMBODIMENTS First Embodiment

A projector in a first embodiment is explained below with reference tothe drawings.

FIG. 1 is a block diagram showing a schematic configuration of aprojector 100 in this embodiment.

As shown in FIG. 1, the projector 100 functioning as an image projectionsystem integrally includes a control section 10, a storing section 11,an input operation section 12, an image-information input section 13, animage-information correcting section 14 functioning as a correctingsection, an image projecting section 15 functioning as a projectingsection, an imaging section 16, a detecting section 17, and acorrection-information generating section 18. The projector 100 projectsan image from the image projecting section 15 onto a projection surfaceSp on the basis of image information input to the image-informationinput section 13.

The projector 100 in this embodiment is capable of correcting distortionof an image that occurs when the image is projected onto the projectionsurface Sp from an oblique direction and correcting distortion of animage that occurs when the image is projected onto the projectionsurface Sp of a three-dimensional shape having unevenness on thesurface. Specifically, the projector 100 projects a predeterminedpattern image Ip (see FIG. 2) from the image projecting section 15 ontothe projection surface Sp and captures the pattern image Ip with theimaging section 16 to thereby recognize distortion of an image andgenerates correction information for correcting the distortion.Thereafter, the projector 100 applies correction processing based on thegenerated correction information on an image that should originally bedisplayed, that is, an image that should be displayed in a state inwhich the distortion is corrected (this image is hereinafter referred toas “display image” as well) and projects an image after the processingonto the projection surface Sp.

The control section 10 includes one or a plurality of processors. Thecontrol section 10 operates according to a control program stored in thestoring section 11 to thereby collectively control the operation of theprojector 100.

The storing section 11 includes a RAM (Random Access Memory), which is avolatile memory, and a ROM (Read Only Memory), which is a nonvolatilememory. The RAM is used for temporary storage of various data and thelike. The ROM stores a control program, control data, and the like forcontrolling the operation of the projector 100. The storing section 11in this embodiment has stored therein image data (pattern image data Dp)corresponding to a pattern image Ip for correcting distortion of animage. The storing section 11 may have stored therein image data for adisplay image.

The input operation section 12 includes a plurality of operation keyswith which a user gives various instructions to the projector 100. Asthe operation keys included in the input operation section 12, thereare, for example, a “power key” for switching ON and OFF of a powersupply, a “menu key” for displaying a setting menu for performingvarious kinds of setting, and a “direction key” for selecting an item ofthe setting menu. When the user operates the various operation keys ofthe input operation section 12, the input operation section 12 outputsan operation signal corresponding to operation content of the user tothe control section 10. Note that a remotely controllable remotecontroller (not shown in FIG. 1) may be used as the input operationsection 12. In this case, the remote controller transmits an operationsignal of an infrared ray corresponding to the operation content of theuser. A not-shown remote-controller-signal receiving section receivesthe operation signal and communicates the operation signal to thecontrol section 10.

The image-information input section 13 is connected to an external imagesupply apparatus (not shown in FIG. 1) such as a computer or an imagereproducing apparatus. The image-information input section 13 receivessupply of image information for a display image from the image supplyapparatus. The image-information input section 13 can receive, from thecontrol section 10, supply of image information (the pattern image dataDp and the image data for a display image) stored in the storing section11. The image-information input section 13 applies, on the basis of thecontrol by the control section 10, various kinds of processing (e.g.,resolution conversion processing and image quality adjustmentprocessing) according to necessity to the image information input fromthe image supply apparatus or the control section 10 and outputs theimage information after the processing to the image-informationcorrecting section 14.

The image-information correcting section 14 corrects, on the basis ofthe control by the control section 10, distortion of an image due toprojection from an oblique direction and distortion of an image due tounevenness and the like of the projection surface Sp. Specifically, theimage-information correcting section 14 applies correction processingbased on correction information input from the control section 10 to theimage information input from the image-information input section 13 andoutputs the image information after the processing to a light-valvedriving section 24 of the image projecting section 15.

The image projecting section 15 is configured by a light source 21,three liquid crystal light valves 22R, 22G, and 22B functioning as lightmodulating devices, a projection lens 23 functioning as a projectionoptical system, a light-valve driving section 24, and the like. Theimage projecting section 15 modulates light emitted from the lightsource 21 with the liquid crystal light valves 22R, 22G, and 22B to formimage light and projects the image light from the projection lens 23 todisplay an image on the projection surface Sp.

The light source 21 includes a light source lamp of a discharge typesuch as an ultra-high pressure mercury lamp or a metal halide lamp or asolid state light source such as a light emitting diode or asemiconductor laser. The light emitted from the light source 21 isconverted into light having a substantially uniform luminancedistribution by a not-shown integrator optical system and separated intocolor light components of red (R), green (G), and blue (B), which arethe three primary colors of light, by a not-shown color separationoptical system. Thereafter, the color light components are respectivelymade incident on the liquid crystal light valves 22R, 22G, and 22B.

The liquid crystal light valves 22R, 22G, and 22B are configured by, forexample, liquid crystal panels of a transmission type, in each of whichliquid crystal is encapsulated between a pair of transparent substrates.In the liquid crystal panels, rectangular image forming regions Aiformed by pluralities of pixels arrayed in a matrix shape are formed. Itis possible to apply a driving voltage to the liquid crystal for each ofthe pixels.

The light-valve driving section 24 forms images in the image formingregions Ai of the liquid crystal light valves 22R, 22G, and 22B.Specifically, the light-valve driving section 24 applies a drivingvoltage corresponding to the image information input from theimage-information correcting section 14 to the pixels of the imageforming regions Ai and sets the pixels to light transmittancecorresponding to the image information. The light emitted from the lightsource 21 is transmitted through the image forming regions Ai of theliquid crystal light valves 22R, 22G, and 22B to thereby be modulatedfor each of the pixels. Image light corresponding to the imageinformation is formed for each of the color lights. The formed colorimage lights are combined for each of the pixels by a not-shown colorcombination optical system to be image light representing a color image.The image light is enlarged and projected onto the projection surface Spby the projection lens 23. As a result, an image based on the imageinformation is displayed on the projection surface Sp.

The imaging section 16 is a camera including an imaging device (notshown in FIG. 1) such as a CCD (Charge Coupled Device) sensor or a CMOS(Complementary Metal Oxide Semiconductor) sensor. The imaging section 16captures, on the basis of the control by the control section 10, thepattern image Ip projected on the projection surface Sp and outputsimage information (captured image information), which is a result of thecapturing, to the detecting section 17.

The detecting section 17 operates according to the control by thecontrol section 10 and detects, on the basis of the captured imageinformation input from the imaging section 16, from the captured patternimage Ip, a plurality of reference points for correcting distortion ofan image. Specifically, the detecting section 17 detects, as referencepoints, vertexes of a plurality of rectangular unit patterns included inthe pattern image Ip. At this time, the detecting section 17 candistinguish colors of the unit patterns and generate, on the basis ofthe distinguished colors of the unit patterns, identificationinformation for identifying the reference points. The detecting section17 outputs coordinates of the reference points on the image (thecaptured image) based on the captured image information and thegenerated identification information to the correction-informationgenerating section 18.

The correction-information generating section 18 operates according tothe control by the control section 10 and recognizes distortion of theimage based on the coordinates of the reference points and theidentification information input from the detecting section 17. Thecorrection-information generating section 18 generates, on the basis ofthe control by the control section 10, correction information forcorrecting the recognized distortion of the image and outputs thegenerated correction information to the control section 10.

Note that the image-information input section 13, the image-informationcorrecting section 14, the detecting section 17, and thecorrection-information generating section 18 may be configured by one ora plurality of processors and the like or may be configured by adedicated processing device such as an ASIC (Application SpecificIntegrated Circuit) or an FPGA (Field Programmable Gate Array).

FIG. 2 is a diagram showing an image based on the pattern image Ip forcorrecting distortion of an image, that is, the pattern image data Dpstored in the storing section 11.

As shown in FIG. 2, the pattern image Ip is a rectangular image formedin the image forming region Ai and is an image on which a plurality ofunit patterns Ur, Ug, Uc, Um, Uy, and Uk having different colors(wavelength regions) are arrayed in a 15×9 matrix shape. Specifically,the pattern image Ip includes unit patterns of seven colors, that is,red unit patterns Ur, green unit patterns Ug, blue unit patterns Ub,cyan unit patterns Uc, magenta unit patterns Um, yellow unit patternsUy, and black unit patterns Uk. All of shapes of the unit patterns arerectangular (in this embodiment, square). Note that, in the followingexplanation, the unit patterns are collectively referred to as “unitpatterns U” as well.

The color (red) of the unit patterns Ur is a color obtained bymaximizing the light transmittance of the liquid crystal light valve 22Rfor red light and minimizing the light transmittance of the liquidcrystal light valves 22G and 22B for green light and blue light. Thecolor (green) of the unit patterns Ug is a color obtained by maximizingthe light transmittance of the liquid crystal light valve 22G for greenlight and minimizing the light transmittance of the liquid crystal lightvalves 22R and 22B for red light and blue light. The color (blue) of theunit patterns Ub is a color obtained by maximizing the lighttransmittance of the liquid crystal valve 22B for blue light andminimizing the light transmittance of the liquid crystal light valves22R and 22G for red light and green light. The color (cyan) of the unitpatterns Uc is a color obtained by maximizing the transmittance of theliquid crystal light valves 22G and 22B for green light and blue lightand minimizing the light transmittance of the liquid crystal light valve22R for red light. The color (magenta) of the unit patterns Um is acolor obtained by maximizing the light transmittance of the liquidcrystal light valves 22R and 22B for red light and blue light andminimizing the light transmittance of the liquid crystal light valve 22Gfor green light. The color (yellow) of the unit patterns Uy is a colorobtained by maximizing the light transmittance of the liquid crystallight valves 22R and 22G for red light and green light and minimizingthe light transmittance of the liquid crystal light valve 22B for bluelight. The color (black) of the unit patterns Uk is a color obtained byminimizing the light transmittance of the liquid crystal light valves22R, 22G, and 22B for red light, green light, and blue light.

In this embodiment, the detecting section 17 of the projector 100detects, from the pattern image Ip captured by the imaging section 16,as reference points, vertexes (corners) of the unit patterns U of sixcolors excluding the black unit patterns Uk. Therefore, in the followingexplanation, the unit patterns U of the six colors other than the blackunit patterns Uk are referred to as “unit patterns for detection U” aswell. The unit patterns for detection U are arranged in a checkeredpattern not to be adjacent to one another in a longitudinal directionand a lateral direction. The black unit patterns Uk are arranged inpositions other than positions where the unit patterns for detection Uare arranged. That is, the black unit patterns Uk are patternsfunctioning as backgrounds of the unit patterns for detection U. Thepattern image Ip can also be regarded as a lattice-like image formed bycontour lines of the unit patterns U. In this case, intersections of alattice are reference points. Note that the pattern image Ip shown inFIG. 2 includes 16×10 (160) reference points arrayed in a matrix shape.Among the reference points, reference signs are attached to only sixteenreference points in the top row (reference points C1 to C16 in orderfrom the left).

In FIG. 2, twelve unit patterns U are arranged in a 4×3 matrix shape ina rectangular range indicated by an alternate long and short dash line.In the following explanation, a set of the twelve unit patterns U isreferred to as “basic pattern group V”. In the basic pattern group V,the unit patterns for detection U, one each of which is for each of thecolors, are arranged in a predetermined array. Six black unit patternsUk are arranged to be adjacent to the unit patterns for detection U. Thepattern image Ip is an image including a plurality of basic patterngroups V. In other words, the pattern image Ip is a part of an image onwhich a plurality of the same basic pattern groups V are arrayedlongitudinally and laterally. Note that, in the pattern image Ip shownin FIG. 2, the basic pattern groups V are repeatedly arrayed in thelateral direction and the basic pattern groups V adjacent to one anotherin the longitudinal direction are arranged to be shifted by one column(by one unit pattern U).

The operation of the projector 100 is explained.

FIG. 3 is a flowchart for explaining the operation (a control method) ofthe projector 100 in performing distortion correction.

When the user operates the input operation section 12 of the projector100 to instruct a start of the distortion correction, the controlsection 10 of the projector 100 starts operation conforming to a controlprogram for performing the distortion correction.

As shown in FIG. 3, in step S101, the control section 10 causes theimage projecting section 15 to project the pattern image Ip.Specifically, the control section 10 reads out the pattern image data Dpfrom the storing section 11 and outputs the pattern image data Dp to theimage-information input section 13. The control section 10 instructs theimage-information correcting section 14 not to perform correctionprocessing. As a result, the pattern image Ip is formed over the entireimage forming region Ai and projected onto the projection surface Sp.When the image projecting section 15 is not right opposed to theprojection surface Sp and when unevenness is present on the projectionsurface Sp, the pattern image Ip is distorted and displayed on theprojection surface Sp.

In step S102, the control section 10 causes the image pickup section 16to capture the pattern image Ip projected on the projection surface Sp.

Subsequently, in step S103, the control section 10 instructs thedetecting section 17 to detect a plurality of reference points from acaptured image obtained by the imaging section 16 capturing the patternimage Ip. As explained above, the detecting section 17 detects thevertexes of the unit patterns for detection U as reference points,generates identification information for identifying the referencepoints, and outputs coordinates of the reference points in the capturedimage and the identification information to the correction-informationgenerating section 18.

In step S104, the control section 10 determines whether the detectingsection 17 has successfully detected all the reference points. When allthe reference points are detected, the control section 10 shifts theprocessing to step S105. When a part of the reference points are notdetected, the control section 10 shifts the processing to step S106.

When all the reference points are detected and the processing shifts tostep S105, the control section 10 instructs the correction-informationgenerating section 18 to generate correction information usingcoordinates of all the detected reference points. Thereafter, in stepS107, the control section 10 instructs the image-information inputsection 13 to output image information for a display image, outputs thegenerated correction information to the image-information correctingsection 14, causes the image-information correcting section 14 to startcorrection of the display image, and ends the processing.

FIGS. 4A to 4D are diagrams for explaining distortion correctionprocessing performed when all the reference points are detected. FIG. 4Ais a plan view of a vertically disposed projection surface Sp viewedfrom above. FIG. 4B is a diagram showing a captured image Ic obtained byimaging the projection surface Sp, on which the pattern image Ip isprojected, with the imaging section 16. FIG. 4C is a diagram showing acorrection region Ac formed in the image forming region Ai. FIG. 4D is adiagram showing a state in which a display image Id is projected ontothe projection surface Sp in a distortion corrected state.

For example, as shown in FIG. 4, when the projector 100 is set such thatan optical axis Ax of the image projecting section 15 obliquely crossesthe projection surface Sp, the pattern image Ip is distorted in atrapezoidal shape and projected (step S101). As shown in FIG. 4B, thecaptured image Ic including the pattern image Ip in a distorted state iscaptured by the imaging section 16 (step S102). Reference points aredetected from the captured image Ic by the detecting section 17 (stepS103).

When all the reference points are detected (Yes in step S104), thecorrection-information generating section 18 recognizes distortion foreach of the unit patterns U on the basis of coordinates of the detectedreference points and generates correction information for correcting thedistortion. Specifically, coordinates (reference coordinates) ofreference points in the rectangular pattern image Ip are known andstored in the storing section 11 in advance. The correction-informationgenerating section 18 sets, in the image forming region Ai, on the basisof a correspondence relation between the coordinates (detectedcoordinates) of the reference points detected on the captured image Icby the detecting section 17 and the reference coordinates, as shown inFIG. 4C, the correction region Ac formed by a plurality of blocks B of ashape with which only the distortion of the unit patterns U is offset.The correction-information generating section 18 generates, on the basisof the reference coordinates of the reference points and coordinates ofvertexes of the blocks B in the correction region Ac, as the correctioninformation, a conversion table for deforming (projection-converting)the display image in block B units and outputs the conversion table tothe control section 10 (step S105).

The control section 10 outputs the input correction information to theimage-information correcting section 14 and causes the image-informationcorrecting section 14 to start correction of the image information onthe basis of the correction information (step S107). Thereafter, theimage-information correcting section 14 performs, on image informationfor a display image sequentially input from the image-information inputsection 13, correction processing for deforming the display image inblock B units and performs processing for setting a region Ao on theouter side of the correction region Ac to black. As a result, as shownin FIG. 4D, the display image Id is projected in a state in which thedistortion is corrected when viewed from the direction of the imagingsection 16. Note that, in this embodiment, the distortion is correctedin block B units on the basis of a large number of reference pointsarrayed in a matrix shape. Therefore, even when small unevenness or thelike is present on the projection surface Sp, it is also possible tocorrect distortion due to the unevenness or the like.

Referring back to FIG. 3, in step S104, when a part of the referencepoints are not detected and the processing shifts to step S106, thecontrol section 10 instructs the correction-information generatingsection 18 to generate correction information using only a detected partof the reference points. Thereafter, in step S107, the control section10 instructs the image-information input section 13 to output the imageinformation for the display image, outputs the generated correctioninformation to the image-information correcting section 14, causes theimage-information correcting section 14 to start correction of thedisplay image, and ends the processing.

FIGS. 5A to 5E are diagrams for explaining distortion correctionprocessing performed when a part of the reference points are notdetected. FIG. 5A is a plan view of the vertically disposed projectionsurface Sp viewed from above. FIG. 5B is a diagram showing the capturedimage Ic obtained by imaging, with the imaging section 16, theprojection surface Sp on which the pattern image Ip is projected. FIG.5C is a diagram showing a first correction region Ac1 and a secondcorrection region Ac2 formed in the image forming region Ai. FIG. 5D isa diagram for explaining correction processing by the image-informationcorrecting section 14. FIG. 5E is a diagram showing a state in which thedisplay image Id is projected onto the projection surface Sp in adistortion corrected state.

For example, as shown in FIG. 5A, when a relatively large step(unevenness) is present on the projection surface Sp and the projector100 projects an image to overlap the step, a situation could occur inwhich a part of reference points are blocked by the step and cannot beimaged because of a parallax between the image projecting section 15 andthe imaging section 16. FIG. 5A shows, when reference points disposed inthe top row among the reference points included in the pattern image Ipare represented as reference points C1 to C16 (see FIG. 2) in order fromthe left, in which positions of the projection surface Sp the referencepoints C1 to C16 are projected. As shown in FIG. 5A, one reference pointC7 among the reference points C1 to C16 is projected onto a blind areaof the imaging section 16 and cannot be imaged by the imaging section 16(see FIG. 5B). On the pattern image Ip, the same applies to other ninereference points present below the reference point C7. The detectingsection 17 cannot detect ten reference points in total (No in stepS104).

In such a case, if the pattern image Ip is a simple checkered pattern oftwo colors of white and black, it is difficult to distinguish, among alarge number of reference points, which reference point is detected andwhich reference point is not detected. That is, because a correctionrelation between the reference coordinates of the reference points andthe detected coordinates is unclear, the correction-informationgenerating section 18 cannot generate correction information.

On the other hand, in this embodiment, when detecting the referencepoints, the detecting section 17 distinguishes a color of a detectiontarget unit pattern U and generates, on the basis of the distinguishedcolor, the positions of detection target vertexes in the unit pattern U,and the like, identification information for identifying the referencepoints. The detecting section 17 outputs the generated identificationinformation to the correction-information generating section 18 togetherwith coordinates.

The correction-information generating section 18 can identify thereference points on the basis of the identification information andspecify to which vertexes in the basic pattern group V the referencepoints correspond. That is, the correction-information generatingsection 18 can estimate original positions (positions on the rectangularpattern image Ip) of the detected reference points. Therefore, acorresponding relation between the reference coordinates and thedetected coordinates is clarified. As a result, thecorrection-information generating section 18 can generate correctioninformation using only a detected part of the reference points.

Specifically, as shown in FIG. 5C, the correction-information generatingsection 18 sets, on the basis of reference points detected from theprojection surface Sp on the left side of the step, in the image formingregion Ai, the first correction region Ac1 formed by a plurality ofblocks B of a shape with which the distortion of the unit patterns U isoffset. Further, the correction-information generating section 18 sets,on the basis of reference points detected from the projection surface Spon the right side of the step, the second correction region Ac2 alsoformed by a plurality of blocks B. The correction-information generatingsection 18 generates, on the basis of the reference coordinates of thereference points and coordinates of vertexes of the blocks B, as thecorrection information, a conversion table for deforming(projection-converting) the display image in block B units and outputsthe conversion table to the control section 10 (step S106).

The control section 10 outputs the input correction information to theimage-information correcting section 14 and causes the image-informationcorrecting section 14 to start correction of the image information onthe basis of the correction information (step S107). Thereafter, asshown in FIG. 5D, the image-information correcting section 14 performs,on image information for a display image sequentially input from theimage-information input section 13, correction processing for dividingthe display image Id into two and deforming the display image Id inblock B units and performs processing for setting the region Ao on theouter side of the first correction region Ac1 and the second correctionregion Ac2 to black. As a result, as shown in FIG. 5E, the display imageId is projected on both the left and right sides of the step of theprojection surface Sp in a state in which the distortion is correctedwhen viewed from the imaging section 16.

Note that, when the image-information correcting section 14 performs thecorrection processing explained above, the display image Id is dividedinto two parts by the black region Ao. However, the correctionprocessing is not limited to this form. For example, as shown in FIG. 6,the correction-information generating section 18 may generate correctioninformation for allocating a part of the display image Id to, togetherwith the first correction region Ac1 and the second correction regionAc2, a third correction region Ac3 sandwiched by the first correctionregion Ac1 and the second correction region Ac2. With this form,although a part of the display image Id is blocked by the step whenviewed from the direction of the imaging section 16, because the blackregion Ao is absent around the step, the display image Id is notdivided. Alternatively, the image-information correcting section 14 maygenerate, on the basis of an image around a position where the displayimage Id is divided, an image that should be formed in the thirdcorrection region Ac3.

As explained above, with the projector 100 according to this embodiment,effects explained below can be obtained.

(1) With the projector 100 in this embodiment, because the pattern imageIp includes the unit patterns U of seven types (seven colors), it ispossible to increase an interval of arrangement of the unit patterns Uof the same color compared with when unit patterns U of two types likewhite rectangular patterns and black rectangular patterns are used.Therefore, even when a part of the reference points are not detected bythe detecting section 17, it is possible to identify the detectedreference point on the basis of the color of the unit patterns U. As aresult, because a correspondence relation between the positions of thedetected reference points and original positions (positions on therectangular pattern image Ip) of the reference points is clarified, itis possible to appropriately correct an image.

(2) With the projector 100 in this embodiment, because the pattern imageIp includes the plurality of basic pattern groups V, it is possible toreduce a necessary number of colors of the unit patterns U. It is easyto distinguish the colors of the unit patterns U.

(3) With the projector 100 in this embodiment, because the detectingsection 17 detects the vertexes of the unit patterns U as the referencepoints, it is possible to easily detect the reference points.

(4) With the projector 100 in this embodiment, because the imageprojecting section 15 and the imaging section 16 are integrallyconfigured, it is unnecessary to adjust disposition of the imageprojecting section 15 and the imaging section 16 every time.

(5) With the projector 100 in this embodiment, the colors (red, green,blue, cyan, magenta, yellow, and black) of the unit patterns U are setto colors obtained by maximizing or minimizing the light transmittanceof the liquid crystal light valves 22R, 22G, and 22, that is, colors nothaving intermediate gradations. Therefore, it is possible to easilydistinguish the colors. Besides the seven colors, the unit patterns U ofwhite obtained by maximizing the light transmittance of all the liquidcrystal light valves 22R, 22G, and 22B may be used.

Note that the number of colors of the unit patterns U only has to bethree or more. As the number of colors is reduced, it is easier todistinguish the colors with the detecting section 17. However, whennumber of colors of the unit patterns U is reduced, the basic patterngroup V decreases in size and the unit patterns U of the same color arearranged at a small interval. Therefore, when the unit patterns U areidentified, the unit patterns U are likely to be confused withneighboring unit patterns U of the same color. Therefore, the number ofcolors of the unit patterns U is desirably increased as much as possiblewithin a range in which distinction by the detecting section 17 ispossible. Colors having intermediate gradations may be used if thedetecting section 17 can distinguish the colors. All the unit patternsfor detection U included in the pattern image Ip may be set to colorsdifferent from one another.

Second Embodiment

A projector in a second embodiment is explained below with reference tothe drawings.

When performing processing for correcting distortion, the projector 100in this embodiment projects the pattern image Ip different from thepattern image Ip projected by the projector 100 in the first embodiment.

FIG. 7 is a diagram showing the pattern image Ip in this embodiment.

As shown in FIG. 7, the pattern image Ip in this embodiment includes aplurality of rectangular (in this embodiment, square) unit patterns U.The plurality of unit patterns U include unit patterns Uw entirelycolored in white, unit patterns Uk entirely colored in black, and unitpatterns Ut of two colors obtained by combining white triangles andblack triangles. The unit pattern Ut of two colors is the unit pattern Uin which one of two triangles (in this embodiment, isosceles righttriangles) generated by dividing the rectangular unit pattern U with adiagonal line is colored in white and the other is colored in black. Theunit patterns U can be further classified into four according to apositional relation between the white triangle and the black triangle(e.g., according to in which position of the upper left, the upperright, the lower left, and the lower right the white triangle islocated). That is, the pattern image Ip in this embodiment is configuredby unit patterns U of six types having different patterns.

In this embodiment, the detecting section 17 of the projector 100detects, as reference points, vertexes of the unit pattern Uw entirelycolored in while and vertexes of white triangles of the unit patterns Utof two colors. That is, unit patterns U of five types including the unitpattern Uw entirely colored in white (one type) and unit patterns Ut oftwo colors (four types) are used as the unit patterns for detection U.The pattern image Ip is an image on which a plurality of rectangularunit patterns U are arranged in a matrix shape longitudinally andlaterally. The unit patterns U are disposed such that vertexes of allthe unit patterns U are detected as reference points of any one of theunit patterns for detection U. That is, a plurality of reference pointsare also arranged in a matrix shape.

In FIG. 7, a rectangular range indicated by an alternate long and shortdash line indicates the basic pattern group V in which a plurality ofunit patterns U are arranged in a predetermined array. The basic patterngroup V does not include the same combination of a plurality of unitpatterns U surrounding detection target reference points (vertexes). Thepattern image Ip is an image including a plurality of basic patterngroups V. In other words, the pattern image Ip is a part of an image onwhich a plurality of the same basic pattern groups V are arrayedlongitudinally and laterally. Note that, in the pattern image Ip shownin FIG. 7, the basic pattern groups V are repeatedly arrayed in thelateral direction. The basic pattern groups V adjacent to one another inthe longitudinal direction are arranged to be shifted by three columns(three unit patterns U).

When detecting reference points, the detecting section 17 generatesidentification information on the basis of a type (a pattern) of adetection target unit pattern U, a position of a detection target vertexin the unit pattern U, types (patterns) of the other unit patterns Usurrounding the vertex, and the like and outputs the identificationinformation to the correction-information generating section 18 togetherwith coordinates. Therefore, even when a part of the reference pointsare not detected because of unevenness (a step, etc.) of the projectionsurface Sp, the correction-information generating section 18 canestimate original positions (positions on the rectangular pattern imageIp) of the reference points on the basis of the identificationinformation. A correspondence relation between reference coordinates anddetected coordinates is clarified. As a result, thecorrection-information generating section 18 can generate correctioninformation using only a detected part of the reference points.

As explained above, with the projector 100 in this embodiment, thepattern image Ip includes the five unit patterns for detection U havingthe different patterns. Therefore, it is possible to increase aninterval of arrangement of the unit patterns U of the same typescompared with when the unit patterns U of two types like a whiterectangular pattern and a black rectangular pattern are used. Therefore,even when a part of the reference points are not detected by thedetecting section 17, it is possible to identify detected referencepoints on the basis of the patterns and the like of the unit patterns U.As a result, a correspondence relation between the positions of thedetected reference points and original positions (positions on therectangular pattern image Ip) of the reference points is clarified.Therefore, it is possible to appropriately correct an image.

Note that the number of types of the unit patterns U only has to bethree or more. However, as in the first embodiment, it is desirable toincrease the number as much as possible within a range in whichdistinction by the detecting section 17 is possible.

Third Embodiment

A projector according to a third embodiment is explained below withreference to the drawings.

When performing processing for correcting distortion, the projector 100in this embodiment projects the pattern image Ip different from thepattern image Ip projected by the projector 100 in the embodimentsexplained above.

FIG. 8 is a diagram showing the pattern image Ip in this embodiment.

As shown in FIG. 8, the pattern image Ip in this embodiment is an imageon which a plurality of unit patterns U (Ur, Ug, Ub, Uc, Um, and Uy)having luminance distributions are arranged in a matrix shapelongitudinally and laterally. The plurality of unit patterns U includethe unit patterns U of six colors, that is, red unit patterns Ur, greenunit patterns Ug, blue unit patterns Ub, cyan unit patterns Uc, magentaunit patterns Um, and yellow unit patterns Uy. The unit patterns haveluminance distributions in which luminance is the largest in the centersof the luminance distributions. In this embodiment, all the unitpatterns U of the six colors are the unit patterns for detection U.

In FIG. 8, a rectangular range indicated by an alternate long and shortdash line indicates the basic pattern group V in which six unit patternsU are arranged in a 2×3 matrix shape. The unit patterns U, one each ofwhich is for each of the colors, are arranged in a predetermined array.The pattern image Ip is an image including a plurality of basic patterngroups V. In other words, the pattern image Ip is a part of an image onwhich a plurality of the same basic pattern groups V are arrayedlongitudinally and laterally. Note that, in the pattern image Ip shownin FIG. 8, the basic pattern groups V are repeatedly arrayed in thelateral direction and the basic pattern groups V adjacent to one anotherin the longitudinal direction are arranged to be shifted by one column(by one unit pattern U).

In this embodiment, the detecting section 17 of the projector 100detects, from the pattern image Ip captured by the imaging section 16,as reference points, positions where luminance is the largest (maximumluminance positions) in the unit patterns U. In the pattern image Ip,the reference points are arranged in a matrix shape.

When detecting the reference points, the detecting section 17 generatesidentification information on the basis of a color of a detection targetunit pattern U and outputs the identification information to thecorrection-information generating section 18 together with coordinates.Therefore, even when a part of the reference points are not detectedbecause of unevenness (a step, etc.) of the projection surface Sp, it ispossible to estimate original positions (positions on the rectangularpattern image Ip) of the reference points. A correspondence relationbetween reference coordinates and detected coordinates is clarified. Asa result, the correction-information generating section 18 can generatecorrection information using only a detected part of the referencepoints.

Note that the detecting section 17 may be configured to determine themaximum luminance positions on the basis of a rate of change (agradient) of the luminance of regions around the maximum luminancepositions. In this case, even when the maximum luminance positions areblocked by unevenness or the like of the projection surface Sp, it ispossible to estimate the maximum luminance positions.

As explained above, with the projector 100 in this embodiment, becausethe reference points are detected on the basis of the luminance of theunit patterns U, it is possible to accurately detect the referencepoints. In particular, the projector 100 is effective when the vertexesof the rectangular unit patterns U cannot be accurately detected as inthe first embodiment because the spectral reflectance of the projectionsurface Sp is different depending on places and is affected by externallight.

Fourth Embodiment

A projector in a fourth embodiment is explained below with reference tothe drawings.

When performing processing for correcting distortion, the projector 100in this embodiment projects the pattern image Ip different from thepattern image Ip projected by the projector 100 in the embodimentsexplained above.

FIG. 9 is a diagram showing the pattern image Ip in this embodiment.

As shown in FIG. 9, the pattern image Ip is an image on which aplurality of unit patterns U (Ur, Ug, Ub, Uc, Um, and Uy) are arrangedin a matrix shape. The plurality of unit patterns U include the unitpatterns U of seven colors, that is, red unit patterns Ur, green unitpatterns Ug, blue unit patterns Ub, cyan unit patterns Uc, magenta unitpatterns Um, yellow unit patterns Uy, and black unit patterns Uk. Theunit patterns of six colors other than the black unit patterns Uk arethe unit patterns for detection U.

The unit patterns for detection U are arranged in a checkered patternnot to be adjacent to one another. The black unit patterns Uk arearranged in positions other than positions where the unit patterns fordetection U are arranged. As in the first embodiment, the detectingsection 17 of the projector 100 detects vertexes of the unit patternsfor detection U as reference points from the pattern image Ip capturedby the imaging section 16. In the pattern image Ip, the reference pointsare arranged in a matrix shape.

In FIG. 9, a rectangular range indicated by an alternate long and shortdash line indicates the basic pattern group V. The basic pattern group Vin this embodiment is a pattern group in which twelve unit patterns Uare arranged in a row in the lateral direction. In the basic patterngroup V, the unit patterns for detection U, one each of which are foreach of the colors, are arranged in predetermined order alternately withthe black unit patterns Uk. The pattern image Ip is an image including aplurality of basic pattern groups V. In other words, the pattern imageIp is a part of an image on which a plurality of the same basic patterngroups V are arrayed longitudinally and laterally. Note that, in thepattern image Ip shown in FIG. 9, the basic pattern groups V arerepeatedly arrayed in the lateral direction and the basic pattern groupsV adjacent to one another in the longitudinal direction are arranged tobe shifted by one column (by one unit pattern U).

In this embodiment, the unit patterns U are arranged along a pluralityof epipolar lines Le1 decided according to a disposition relationbetween the image projecting section 15 and the imaging section 16.Specifically, the unit patterns U are arranged in a matrix shape in adirection along the epipolar lines Le1 and a direction crossing theepipolar lines Le1. Vertexes of the unit patterns for detection U arelocated on the epipolar lines Le1.

The epipolar lines Le1 are explained.

FIG. 10 is an explanatory diagram for explaining an epipolar geometryconcerning the image projecting section 15 and the imaging section 16.

As shown in FIG. 10, a straight line connecting an optical center O1 ofthe image projecting section 15 and an optical center O2 of the imagingsection 16 is set as a baseline Lb. In an imaginary image plane P1 (aplane equivalent to the image forming region Ai) of the image projectingsection 15, all straight lines passing an intersection (an epipole Pe1)with the baseline Lb are the epipolar lines Le1. When one epipolar lineLe1 is decided in the imaginary image plane P1, an epipolar line Le2corresponding to the epipolar line Le1 is specified on an imaginaryimage plane P2 (a plane equivalent to the captured image Ic) of theimaging section 16. The epipolar line Le2 is a straight line passing anintersection (an epipole Pe2) with the baseline Lb in the imaginaryimage plane P2. Light projected from any position x1 on the epipolarline Le1 of the imaginary image plane P1 onto a position x0 on theprojection surface Sp is projected (imaged) in any position (e.g., aposition x2) on the corresponding epipolar line Le2 on the imaginaryimage plane P2 irrespective of the distance to the projection surfaceSp.

Note that, in FIG. 9, for convenience of explanation, the epipolar linesLe1 are represented by a plurality of parallel lines. However, asexplained above, because all the epipolar lines Le1 are straight linespassing a common epipole Pe1, in general, the epipolar lines Le1radially extend. In this case, the shape of the unit patterns U is not arectangle. However, when an optical axis of the image projecting section15 and an optical axis of the imaging section 16 are parallel, theepipole Pe1 is located at infinity. Therefore, the plurality of epipolarlines Le1 are substantially parallel.

In this way, in this embodiment, the unit patterns for detection U arearranged such that the vertexes are located on the epipolar lines Le1decided in advance. Epipolar lines Le2 on the captured image Iccorresponding to the epipolar lines Le1 can be determined in advanceaccording to the disposition relation between the image projectingsection 15 and the imaging section 16. Therefore, the detecting section17 can detect the reference points by searching for the reference pointson the epipolar lines Le2 determined in advance. When detecting thereference points, the detecting section 17 generates identificationinformation on the basis of the color and the like of the detectiontarget unit pattern U and outputs the identification information to thecorrection-information generating section 18 together with coordinates.Therefore, even when a part of the reference points are not detectedbecause of unevenness (a step, etc.) of the projection surface Sp, thecorrection-information generating section 18 can estimate originalpositions (positions on the rectangular pattern image Ip) of thereference points on the basis of the identification information. Acorrespondence relation between reference coordinates and detectedcoordinates is clarified. As a result, the correction-informationgenerating section 18 can generate correction information using only adetected part of the reference points.

Even when the other unit patterns U of the same color are located nearthe unit patterns for detection U, if the other unit patterns U arepresent in positions away from the search target epipolar line Le2 inthe perpendicular direction, it is unlikely that the unit patterns fordetection U and the other unit patterns U are confused. Therefore, evenwhen the basic pattern group V is formed in a configuration in which theplurality of unit patterns U are arranged in a row along the epipolarline Le1 as in this embodiment, it is possible to easily identify thereference points.

As explained above, with the projector 100 in this embodiment, becausethe detecting section 17 can detect the reference points along theepipolar line Le2, it is easy to detect the reference points.

As explained above, in the direction crossing the epipolar line Le2,even when the unit patterns U of the same type are present in thevicinity, it is unlikely that the unit patterns U are confused.Therefore, in the basic pattern group V, it is possible to array theunit patterns U of the same color in a row along the epipolar line Le1.As a result, it is possible to arrange the unit patterns U of the samecolor apart from one another in the direction along the epipolar lineLe1. Therefore, the unit patterns U are prevented from being confusedwith the other unit patterns U.

Note that the epipolar line Le1 on the pattern image Ip (the imageforming region Ai) is equivalent to a first epipolar line and theepipolar line Le2 on the captured image Ic is equivalent to a secondepipolar line.

Fifth Embodiment

A projector in a fifth embodiment is explained below with reference tothe drawings.

When performing processing for correcting distortion, the projector 100in this embodiment projects the pattern image Ip different from thepattern image Ip projected by the projector 100 in the embodimentsexplained above.

FIG. 11 is a diagram showing the pattern image Ip in this embodiment.

As shown in FIG. 11, the pattern image Ip is an image on which aplurality of belt-like unit patterns U (Ur, Ug, Ub, Uc, Um, Uy, and Uk)arranged to extend in the longitudinal direction (a direction crossingthe epipolar lines Le1) are arrayed in a row in the lateral direction (adirection along the epipolar lines Le1). The plurality of unit patternsU include the unit patterns U of seven colors, that is, red unitpatterns Ur, green unit patterns Ug, blue unit patterns Ub, cyan unitpatterns Uc, magenta unit patterns Um, yellow unit patterns Uy, andblack unit patterns Uk. The unit patterns U of six colors other than theblack unit patterns Uk are the unit patterns for detection U.

In FIG. 11, a rectangular range indicated by an alternate long and shortdash line indicates the basic pattern group V. The basic pattern group Vin this embodiment is a pattern group in which twelve unit patterns Uare arranged in a row in the lateral direction. In the basic patterngroup V, the unit patterns for detection U, one each of which are foreach of the colors, are arranged in predetermined order alternately withthe black unit patterns Uk. The pattern image Ip is a part of an imageon which a plurality of the same basic pattern groups V are arrayed inthe lateral direction.

In this embodiment, end portions of the unit patterns for detection U,that is, points on boundary lines between the unit patterns fordetection U and the black unit patterns Uk are used as reference points.Specifically, intersections of a plurality of epipolar lines Le1 decidedin advance and the end portions (the boundary lines) of the unitpatterns for detection U are used as the reference points. The referencepoints are arranged in a matrix shape in the pattern image Ip (the imageforming region Ai). In the storing section 11, coordinates of thereference points in the pattern image Ip are stored as referencecoordinates. The epipolar lines Le2 on the captured image Iccorresponding to the epipolar lines Le1 can be determined in advanceaccording to a disposition relation between the image projecting section15 and the imaging section 16. Therefore, the detecting section 17 candetect, in the captured image Ic, intersections of the epipolar linesLe2 determined in advance and the end portions of the unit patterns fordetection U as the reference points (detected coordinates).

When detecting the reference points, the detecting section 17 generatesidentification information on the basis of the color and the like of thedetection target unit pattern U and outputs the identificationinformation to the correction-information generating section 18 togetherwith coordinates. Therefore, even when a part of the reference pointsare not detected because of unevenness (a step, etc.) of the projectionsurface Sp, the correction-information generating section 18 canestimate original positions (positions on the rectangular pattern imageIp) of the reference points on the basis of the identificationinformation. A correspondence relation between the reference coordinatesand the detected coordinates is clarified. As a result, thecorrection-information generating section 18 can generate correctioninformation using only a detected part of the reference points.

Note that, in this embodiment, the belt-like unit patterns U arearranged to be orthogonal to the epipolar lines Le1. Therefore, when aplurality of epipolar lines Le1 are not parallel and radially extend,the unit patterns U are formed as arcuate patterns to be orthogonal tothe epipolar lines Le1. However, a relation between the belt-like unitpatterns U and the epipolar lines Le1 only has to be a crossing relationand is not limited to the orthogonal relation.

As explained above, with the projector 100 in this embodiment, as in thefourth embodiment, the detecting section 17 can detect the referencepoints along the epipolar lines Le2. Therefore, it is easy to detect thereference points.

With the projector 100 in this embodiment, the belt-like unit patterns Uare arrayed in a row along the epipolar line Le1. Therefore, it ispossible to arrange the unit patterns U of the same color apart from oneanother in a direction along the epipolar line Le1. Therefore, the unitpatterns U are prevented from being confused with the other unitpatterns U.

With the projector 100 in this embodiment, the detecting section 17detects the end portions of the unit patterns U as the reference points.Therefore, it is possible to easily detect the reference points.

Sixth Embodiment

A projector in a sixth embodiment is explained below with reference tothe drawings.

When performing processing for correcting distortion, the projector 100in this embodiment projects the pattern image Ip different from thepattern image Ip projected by the projector 100 in the embodimentsexplained above.

FIG. 12 is a diagram showing the pattern image Ip in this embodiment.

As shown in FIG. 12, the pattern image Ip in this embodiment is an imageon which a plurality of belt-like unit patterns U (Ur, Ug, Ub, Uc, Um,and Uy) arranged to extend in the longitudinal direction (a directioncrossing the epipolar lines Le1) are arrayed in a row in the lateraldirection (a direction along the epipolar lines Le1). The plurality ofunit patterns U include the unit patterns U of six colors, that is, redunit patterns Ur, green unit patterns Ug, blue unit patterns Ub, cyanunit patterns Uc, magenta unit patterns Um, and yellow unit patterns Uy.All the unit patterns U of six colors are the unit patterns fordetection U. The unit patterns U have luminance distributions. Luminanceis the largest in the center in the direction along the epipolar linesLe1.

In FIG. 12, a rectangular range indicated by an alternate long and shortdash line indicates the basic pattern group V. The basic pattern group Vin this embodiment is a pattern group in which six unit patterns U arearranged in a row in the lateral direction. In the basic pattern groupV, the unit patterns for detection U, one each of which are for each ofthe colors, are arranged in predetermined order. The pattern image Ip isan image including a plurality of basic pattern groups V. In otherwords, the pattern image Ip is a part of an image on which a pluralityof the same basic pattern groups V are arrayed in the lateral direction.

In this embodiment, each positions where the luminance of the unitpatterns U is the largest (maximum luminance positions) on a pluralityof epipolar lines Le1 decided in advance are used as reference points.The reference points are arranged in a matrix shape in the pattern imageIp (the image forming region Ai). Coordinates of the reference points inthe pattern image Ip are stored in the storing section 11 as referencecoordinates. The epipolar lines Le2 on the captured image Iccorresponding to the epipolar lines Le1 can be determined in advanceaccording to a disposition relation between the image projecting section15 and the imaging section 16. Therefore, the detecting section 17 candetect, in the captured image Ic, as reference points (detectedcoordinates), maximum luminance positions of the unit patterns U on theepipolar lines Le2 determined in advance.

When detecting the reference points, the detecting section 17 generatesidentification information on the basis of the color of the detectiontarget unit pattern U and outputs the identification information to thecorrection-information generating section 18 together with coordinates.Therefore, even when a part of the reference points are not detectedbecause of unevenness (a step, etc.) of the projection surface Sp, thecorrection-information generating section 18 can estimate originalpositions (positions on the rectangular pattern image Ip) of thereference points based on the identification information. Acorresponding relation between the reference coordinates and thedetected coordinates is clarified. As a result, thecorrection-information generating section 18 can generate correctioninformation using only a detected part of the reference points.

Note that the detecting section 17 may be configured to determine themaximum luminance positions on the basis of a rate of change (agradient) of the luminance of regions around the maximum luminancepositions. In this case, even when the maximum luminance positions areblocked by unevenness or the like of the projection surface Sp, it ispossible to estimate the maximum luminance positions.

As explained above, with the projector 100 in this embodiment, as in thefourth and fifth embodiments, the detecting section 17 can detect thereference points along the epipolar lines Le2. Therefore, it is easy todetect the reference points.

With the projector 100 in this embodiment, as in the fifth embodiment,because the belt-like unit patterns U are arrayed in a row along theepipolar line Le1, it is possible to arrange the unit patterns U of thesame color apart from one another in a direction along the epipolar lineLe1. Therefore, the unit patterns U are prevented from being confusedwith the other unit patterns U.

With the projector 100 in this embodiment, because the reference pointsare detected on the basis of the luminance of the unit patterns U, it ispossible to accurately detect the reference points. In particular, theprojector 100 is effective when the end portions of the belt-like unitpatterns U cannot be accurately detected as in the fifth embodimentbecause the spectral reflectance of the projection surface Sp isdifferent depending on places and is affected by external light.

Modifications

The embodiments may be changed as explained below.

In the first to sixth embodiments, the projector 100 integrallyincluding the components such as the image projecting section 15 and theimaging section 16 is explained. However, a part or all of thecomponents may be separated. When the image projecting section 15 andthe imaging section 16 are separated, disposition states of the imageprojecting section 15 and the imaging section 16 change every time theprojector 100 is set. Therefore, it is necessary to perform calibration(e.g., processing for associating coordinates on the rectangular patternimage Ip and coordinates on the captured image Ic) every time theprojector 100 is set.

In the first to sixth embodiments, the distortion correction performedwhen a part of the reference points cannot be detected because ofunevenness (a step, etc.) of the projection surface Sp having thethree-dimensional shape is explained. However, the projection surface Spis not limited to the three-dimensional shape. For example, theinvention is also applicable when a part of the reference points cannotbe detected because of a pattern or a stain present on a planarprojection surface Sp, an obstacle, or the like.

In the first to sixth embodiments, the vertexes, end portions, or themaximum luminance positions of the unit patterns U are set as thereference points. However, the reference points are not limited to this.For example, the positions of the centers of gravity of the unitpatterns U may be set as the reference points.

In the first, third, and fourth embodiments, the form is explained inwhich the unit patterns for detection U, one each of which is for eachof the colors, are included in the basic pattern group V. However, aplurality of unit patterns of the same color may be included in thebasic pattern group V if reference points can be identified. Forexample, if a combination of colors of a plurality of unit patterns fordetection U surrounding one reference point (vertex) is differentiatedat all the reference points in the basic pattern group V, even if theplurality of unit patterns of the same color are included, it ispossible to identify the reference points.

In the sixth embodiment, a form may be adopted in which the referencepoints are specified by a phase shift method using the pattern image Ip,a luminance distribution of which in the direction along the epipolarlines Le1 is a sine wave over the plurality of unit patterns U.Specifically, the pattern image Ip (a first pattern image) having theluminance distribution of the sine wave described above is projected andcaptured and the luminance distribution is measured along the epipolarlines Le2 on the captured image Ic. At least two pattern images of asine wave phase-shifted from the first pattern image are projected andluminance distributions are also measured for the respective patternimages along the epipolar lines Le2 on the captured image Ic. Referencepoints (e.g., maximum luminance positions on the first pattern image)may be specified on the basis of results of the measurement of therespective luminance distributions. With this form, even when thereflectance of the projection surface Sp is not uniform, it is possibleto accurately detect the reference points.

In the first and third to sixth embodiments, the pattern image Ipincludes the plurality of unit patterns U having the different colors.However, the difference of the colors is not limited to the differenceof wavelength regions and may include a difference of brightness.

In the third to sixth embodiments, the maximum luminance positions ofthe unit patterns U are set as the reference points. However, positionswhere the luminance is the smallest in the unit patterns U (minimumluminance positions) may be set as the reference points.

In the fourth embodiment, the example of the pattern image Ip on whichthe plurality of unit patterns U having the different colors arearranged along the epipolar lines Le1 is explained. However, the patternimage Ip on which the plurality of unit patterns U having the differentpatterns in the second embodiment are arranged along the epipolar linesLe1 may be adopted.

In the first, fourth, and fifth embodiments, the pattern image Ipincludes the black unit patterns Uk other than the unit patterns fordetection U. However, a configuration not including the black unitpatterns Uk is also possible. However, it is easier to detect thevertexes and the end portions of the unit patterns for detection U whenthe unit patterns for detection U and the black unit patterns Uk arearranged adjacent to each other as in the first, fourth, and fifthembodiments. The basic pattern group V can be increased in size byincluding the black unit patterns Uk in the pattern image Ip. Therefore,it is possible to arrange the unit patterns U of the same color apartfrom one another. The unit patterns U are prevented from being confusedwith the other unit patterns U.

In the fifth and sixth embodiments, each of the belt-like unit patternsU is configured by the single color over the entire region of thebelt-like unit pattern U. However, it is also possible to adopt a formin which a plurality of regions having different colors are arrayed inthe longitudinal direction (the direction crossing the epipolar linesLe1).

In the first to sixth embodiments, when a part of an image projectedfrom the projector 100 is blocked by unevenness of the projectionsurface Sp and a shadow is formed, it is also possible to cause aplurality of projectors 100 to respectively project the image fromdifferent directions to prevent a region where the image is notdisplayed (a region where a shadow is formed) from being generated.

In the first to sixth embodiments, the liquid crystal light valves 22R,22G, and 22B of the transmission type are used as the light modulatingdevices. However, it is also possible to use a light modulating deviceof a reflection type such as a liquid crystal light valve of thereflection type. It is also possible to use a digital mirror device orthe like that modulates light emitted from the light source 21 bycontrolling an emitting direction of incident light for each ofmicromirrors functioning as pixels. The invention is not limited to aconfiguration including a plurality of light modulating devices forrespective color lights and may be a configuration for modulating aplurality of color lights with one light modulating device in atime-division manner.

The entire disclosure of Japanese Patent Application No. 2017-168291,filed on Sep. 1, 2017 is expressly incorporated by reference herein.

What is claimed is:
 1. An image projection system comprising: a projecting section configured to project an image onto a projection surface; a control section configured to cause the projecting section to project a pattern image; an imaging section configured to capture the pattern image projected on the projection surface; a detecting section configured to detect reference points on the basis of the pattern image captured by the imaging section; and a correcting section configured to correct, on the basis of positions of the reference points detected by the detecting section, the image projected by the projecting section, wherein the pattern image includes a plurality of unit patterns for specifying the reference points, and the plurality of unit patterns include at least three types unit patterns which at least one of colors and patterns are different.
 2. The image projection system according to claim 1, wherein the at least three types unit patterns have colors different from one another.
 3. The image projection system according to claim 1, wherein the at least three types unit patterns have patterns different from one another.
 4. The image projection system according to claim 1, wherein the unit patterns are patterns of a shape having vertexes, and the detecting section detects the vertexes of the unit patterns as the reference points.
 5. The image projection system according to claim 1, wherein the unit patterns are patterns having luminance distributions, and the detecting section detects the reference points on the basis of luminance of the unit patterns.
 6. The image projection system according to claim 1, wherein the pattern image includes a plurality of basic pattern groups in which the at least three types unit patterns are arranged in a predetermined array.
 7. The image projection system according to claim 1, wherein in the pattern image, the unit patterns are arranged such that the reference points are located along a first epipolar line determined on the basis of a disposition relation between the projecting section and the imaging section, and the detecting section detects, from the image captured by the imaging section, the reference points along a second epipolar line corresponding to the first epipolar line.
 8. The image projection system according to claim 2, wherein the unit patterns are belt-like patterns extending in a direction crossing a first epipolar line determined on the basis of a disposition relation between the projecting section and the imaging section, and the detecting section detects, from the image captured by the imaging section, the reference points along a second epipolar line corresponding to the first epipolar line.
 9. The image projection system according to claim 8, wherein the detecting section detects end portions of the unit patterns as the reference points.
 10. The image projection system according to claim 8, wherein the unit patterns are patterns having luminance distributions, and the detecting section detects the reference points on the basis of luminance of the unit patterns.
 11. The image projection system according to claim 1, wherein the projecting section and the imaging section are integrally configured.
 12. A control method for an image projection system comprising: projecting a pattern image from a projecting section configured to project an image onto a projection surface; capturing the pattern image projected on the projection surface; detecting reference points on the basis of the captured pattern image; and correcting, on the basis of positions of the detected reference points, the image projected by the projecting section, wherein the pattern image includes a plurality of unit patterns for specifying the reference points, and the plurality of unit patterns include at least three types unit patterns which at least one of colors and patterns are different. 